42 research outputs found
MultiWOZ 2.4: A Multi-Domain Task-Oriented Dialogue Dataset with Essential Annotation Corrections to Improve State Tracking Evaluation
The MultiWOZ 2.0 dataset has greatly stimulated the research of task-oriented
dialogue systems. However, its state annotations contain substantial noise,
which hinders a proper evaluation of model performance. To address this issue,
massive efforts were devoted to correcting the annotations. Three improved
versions (i.e., MultiWOZ 2.1-2.3) have then been released. Nonetheless, there
are still plenty of incorrect and inconsistent annotations. This work
introduces MultiWOZ 2.4, which refines the annotations in the validation set
and test set of MultiWOZ 2.1. The annotations in the training set remain
unchanged (same as MultiWOZ 2.1) to elicit robust and noise-resilient model
training. We benchmark eight state-of-the-art dialogue state tracking models on
MultiWOZ 2.4. All of them demonstrate much higher performance than on MultiWOZ
2.1
ASSIST: Towards Label Noise-Robust Dialogue State Tracking
The MultiWOZ 2.0 dataset has greatly boosted the research on dialogue state
tracking (DST). However, substantial noise has been discovered in its state
annotations. Such noise brings about huge challenges for training DST models
robustly. Although several refined versions, including MultiWOZ 2.1-2.4, have
been published recently, there are still lots of noisy labels, especially in
the training set. Besides, it is costly to rectify all the problematic
annotations. In this paper, instead of improving the annotation quality
further, we propose a general framework, named ASSIST (lAbel noiSe-robuSt
dIalogue State Tracking), to train DST models robustly from noisy labels.
ASSIST first generates pseudo labels for each sample in the training set by
using an auxiliary model trained on a small clean dataset, then puts the
generated pseudo labels and vanilla noisy labels together to train the primary
model. We show the validity of ASSIST theoretically. Experimental results also
demonstrate that ASSIST improves the joint goal accuracy of DST by up to
on MultiWOZ 2.0 and on MultiWOZ 2.4, compared to using only
the vanilla noisy labels
Enhancing Conversational Search: Large Language Model-Aided Informative Query Rewriting
Query rewriting plays a vital role in enhancing conversational search by
transforming context-dependent user queries into standalone forms. Existing
approaches primarily leverage human-rewritten queries as labels to train query
rewriting models. However, human rewrites may lack sufficient information for
optimal retrieval performance. To overcome this limitation, we propose
utilizing large language models (LLMs) as query rewriters, enabling the
generation of informative query rewrites through well-designed instructions. We
define four essential properties for well-formed rewrites and incorporate all
of them into the instruction. In addition, we introduce the role of rewrite
editors for LLMs when initial query rewrites are available, forming a
"rewrite-then-edit" process. Furthermore, we propose distilling the rewriting
capabilities of LLMs into smaller models to reduce rewriting latency. Our
experimental evaluation on the QReCC dataset demonstrates that informative
query rewrites can yield substantially improved retrieval performance compared
to human rewrites, especially with sparse retrievers.Comment: 22 pages, accepted to EMNLP Findings 202
Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking
Dialogue State Tracking (DST) aims to keep track of users’ intentions during the course of a conversation. In DST, modelling the relations among domains and slots is still an under-studied problem. Existing approaches that have considered such relations generally fall short in: (1) fusing prior slot-domain membership relations and dialogue-aware dynamic slot relations explicitly, and (2) generalizing to unseen domains. To address these issues, we propose a novel Dynamic Schema Graph Fusion Network (DSGFNet), which generates a dynamic schema graph to explicitly fuse the prior slot-domain membership relations and dialogue-aware dynamic slot relations. It also uses the schemata to facilitate knowledge transfer to new domains. DSGFNet consists of a dialogue utterance encoder, a schema graph encoder, a dialogue-aware schema graph evolving network, and a schema graph enhanced dialogue state decoder. Empirical results on benchmark datasets (i.e., SGD, MultiWOZ2.1, and MultiWOZ2.2), show that DSGFNet outperforms existing methods
Outlier-Resilient Web Service QoS Prediction
The proliferation of Web services makes it difficult for users to select the
most appropriate one among numerous functionally identical or similar service
candidates. Quality-of-Service (QoS) describes the non-functional
characteristics of Web services, and it has become the key differentiator for
service selection. However, users cannot invoke all Web services to obtain the
corresponding QoS values due to high time cost and huge resource overhead.
Thus, it is essential to predict unknown QoS values. Although various QoS
prediction methods have been proposed, few of them have taken outliers into
consideration, which may dramatically degrade the prediction performance. To
overcome this limitation, we propose an outlier-resilient QoS prediction method
in this paper. Our method utilizes Cauchy loss to measure the discrepancy
between the observed QoS values and the predicted ones. Owing to the robustness
of Cauchy loss, our method is resilient to outliers. We further extend our
method to provide time-aware QoS prediction results by taking the temporal
information into consideration. Finally, we conduct extensive experiments on
both static and dynamic datasets. The results demonstrate that our method is
able to achieve better performance than state-of-the-art baseline methods.Comment: 12 pages, to appear at the Web Conference (WWW) 202
New Approaches in Multi-View Clustering
Many real-world datasets can be naturally described by multiple views. Due to this, multi-view learning has drawn much attention from both academia and industry. Compared to single-view learning, multi-view learning has demonstrated plenty of advantages. Clustering has long been serving as a critical technique in data mining and machine learning. Recently, multi-view clustering has achieved great success in various applications. To provide a comprehensive review of the typical multi-view clustering methods and their corresponding recent developments, this chapter summarizes five kinds of popular clustering methods and their multi-view learning versions, which include k-means, spectral clustering, matrix factorization, tensor decomposition, and deep learning. These clustering methods are the most widely employed algorithms for single-view data, and lots of efforts have been devoted to extending them for multi-view clustering. Besides, many other multi-view clustering methods can be unified into the frameworks of these five methods. To promote further research and development of multi-view clustering, some popular and open datasets are summarized in two categories. Furthermore, several open issues that deserve more exploration are pointed out in the end
Turn-Level Active Learning for Dialogue State Tracking
Dialogue state tracking (DST) plays an important role in task-oriented
dialogue systems. However, collecting a large amount of turn-by-turn annotated
dialogue data is costly and inefficient. In this paper, we propose a novel
turn-level active learning framework for DST to actively select turns in
dialogues to annotate. Given the limited labelling budget, experimental results
demonstrate the effectiveness of selective annotation of dialogue turns.
Additionally, our approach can effectively achieve comparable DST performance
to traditional training approaches with significantly less annotated data,
which provides a more efficient way to annotate new dialogue data.Comment: EMNLP 2023 Main Conferenc
Lending Interaction Wings to Recommender Systems with Conversational Agents
Recommender systems trained on offline historical user behaviors are
embracing conversational techniques to online query user preference. Unlike
prior conversational recommendation approaches that systemically combine
conversational and recommender parts through a reinforcement learning
framework, we propose CORE, a new offline-training and online-checking paradigm
that bridges a COnversational agent and REcommender systems via a unified
uncertainty minimization framework. It can benefit any recommendation platform
in a plug-and-play style. Here, CORE treats a recommender system as an offline
relevance score estimator to produce an estimated relevance score for each
item; while a conversational agent is regarded as an online relevance score
checker to check these estimated scores in each session. We define uncertainty
as the summation of unchecked relevance scores. In this regard, the
conversational agent acts to minimize uncertainty via querying either
attributes or items. Based on the uncertainty minimization framework, we derive
the expected certainty gain of querying each attribute and item, and develop a
novel online decision tree algorithm to decide what to query at each turn.
Experimental results on 8 industrial datasets show that CORE could be
seamlessly employed on 9 popular recommendation approaches. We further
demonstrate that our conversational agent could communicate as a human if
empowered by a pre-trained large language model.Comment: NeurIPS 202
A circular economy approach by co-gasification of water hyacinth and algae bloom for high-quality biochar production
Water hyacinth is of interest for biochar production due to its high biomass yields, high carbon content and environmental benefits of carbon sequestration and pollutants removal. Gasification technology has attracted considerable attention to design a renewable biochar production process to be performed on a larger scale for both separation and immobilization of contaminants from water hyacinth and the production of energy and multifunctional materials. The concept of the circular economy has become popular since it is a solution that will allow countries, firms and consumers to reduce harm to the environment and to close the loop of the product lifecycle through three main approaches of reusing, reducing and recycling materials, energy and waste. This study is focused on the sustainable management of water hyacinth biomass via gasification (300-900ËšC) to high-value products of biochar, bio-oil and syngas, from the perspective of energy consumption, heat reduction and recycling, emissions to the air and residues in the biochar based on circular economy towards environmental sustainability. The objective is to compare two different types of processes of mono-gasification and co-gasification for environmental, economic and social benefits. The environment, economy and society are inter-related to highlight the new insights for the biochar utilization and resonate with phytobioremediation strategy. The process is based on lab-scale gasifiers/pyrolyzers and a functional unit of a 20KW downdraft gasifier. In our previous experience, we have successfully converted waste biomass from horse manure in Singapore Turf Club to syngas and biochar in the downdraft gasifier. In this study, an equipment level of optimization is implemented for best-operating conditions to improve energy efficiency. In the first process of mono-gasification, biochar is produced from water hyacinth. The alternative to this method is co-gasification, where biochar is now produced with addition of algae bloom. The cost-benefit analysis and life cycle analysis demonstrate the difference in sustainability between these two processes, which offers a higher understanding of biochar production and hence determine which method would be the preferable sustainable practice. It is expected that the co-gasification process could increase the syngas, heat and energy production with high-quality biochar production. One of the major challenges is to guarantee the water hyacinth resources are conserved and used efficiently and affordably